[rush] Add PNPM global virtual store support for rush install#5830
[rush] Add PNPM global virtual store support for rush install#5830EscapeB wants to merge 6 commits into
Conversation
96b4d3e to
5f2c85a
Compare
5f2c85a to
7a0be79
Compare
|
I am at the JSNation conference this week, will take a look later. Thanks! |
| * It is not currently compatible with the | ||
| * `usePnpmSyncForInjectedDependencies` experiment. | ||
| * | ||
| * PNPM documentation: https://pnpm.io/settings#enableglobalvirtualstore |
There was a problem hiding this comment.
According to these docs:
If pnpm detects that it is running in CI, this setting is automatically disabled.
🤔 I'm a bit curious how that works. It sounds like it could be nondeterministic "magic" that could make it difficult to repro bugs. If so, Rush usually tries to find a way to disable these behaviors, or replace them with some documented Rush mechanism.
There was a problem hiding this comment.
According to "install" command doc, seems this package(ci-info) was used to detect CI environment

…nfig.json Co-authored-by: Pete Gonzalez <[email protected]>
Co-authored-by: Pete Gonzalez <[email protected]>
Co-authored-by: Pete Gonzalez <[email protected]>
| `${this.rushConfiguration.commonRushConfigFolder}/${RushConstants.pnpmConfigFilename}`, | ||
| rushJsonFolder: this.rushConfiguration.rushJsonFolder, | ||
| pnpmStore: this.rushConfiguration.pnpmOptions.pnpmStore, | ||
| pnpmStorePath: this.rushConfiguration.pnpmOptions.pnpmStorePath, |
There was a problem hiding this comment.
this.rushConfiguration.pnpmOptions.pnpmStore reads from common/config/rush/pnpm-config.json, however each subspace can have its own pnpm-config.json. Each of these files can have its own enableGlobalVirtualStore setting, which complicates the consistency check being performed here.
It seems to me that enableGlobalVirtualStore is probably a machine-specific setting that does NOT affect the pnpm-lockfile.yaml (@EscapeB could you confirm this?). In other words, it determines how where the node_modules symlinks point, but does NOT affect the semantics of the installation plan. If so, then it is very similar to RUSH_PNPM_STORE_PATH.
🚩 If it is a personal preference, then maybe it should not be represented as a setting in pnpm-config.json, as this file is tracked by Git and affects everyone who works in the monorepo. Compare with this alternative design:
- If the local machine defines
RUSH_PNPM_STORE_GLOBAL=1, thenRUSH_PNPM_STORE_PATHwill be used as the global virtual store. (IfRUSH_PNPM_STORE_PATHis empty or unset, then we use PNPM's default "global" store location.) rush installimplements it by emittingenableGlobalVirtualStore=trueintotemp/pnpm-workspace.yaml- Rush (or PNPM) validates that the environment variables don't cause problems or nondeterminism:
- Print an error is
RUSH_PNPM_STORE_GLOBALis used when PNPM version < 10.12.1. - Extend the
RUSH_PNPM_STORE_PATHvalidation so that an error is reported ifRUSH_PNPM_STORE_GLOBALorRUSH_PNPM_STORE_PATHhas changed since the lastrush install. (We don't silently "heal", because this is specifically a performance setting, and it's a real performance problem if different commands are invoked with conflicting environments.)
- Print an error is
There was a problem hiding this comment.
It seems to me that enableGlobalVirtualStore is probably a machine-specific setting that does NOT affect the pnpm-lockfile.yaml
Yes, according to pnpm docs, pnpm-lockfile.yaml won't change, only symlink structure under node_modules changes.
I didn't reliazed there are multiple subspaces and will have multiple pnpm-config.json, I agreed that environment var will be a better approach.
| node: process.versions.node | ||
| }); | ||
|
|
||
| if ( |
There was a problem hiding this comment.
This PR also avoids acquiring the global package manager install lock when the requested package manager is already installed and no matching install lock file exists. If a lock file exists, Rush still acquires the lock to preserve stale/dirty recovery behavior.
@EscapeB this secondary fix is "small", but its review will be entirely focused on proving there aren't any race conditions. Could you extract it into a separate PR so it is easier to analyze? This will also decouple the two fixes, so we can merge+publish one earlier if the other one takes longer to review. 🙏
There was a problem hiding this comment.
Sure, I will create another PR for this small fix.
|
I extracted the package-manager install lock optimization into a separate PR: #5844 This PR has been updated to remove the |
Summary
This PR adds Rush support for PNPM's global virtual store for workspace installs.
This targets workflows where multiple Git worktrees of the same monorepo are created and deleted frequently, especially AI-assisted development workflows where several agents may work in parallel. Without this, each worktree may recreate a large local virtual store under
common/temp/node_modules/.pnpm, making worktree setup and cleanup expensive.When enabled, Rush writes
enableGlobalVirtualStore: trueto the generatedpnpm-workspace.yaml. PNPM then stores package instance folders under the shared PNPM store instead of recreating them inside every worktree.Existing repositories keep their current behavior by default. The feature is enabled per machine/session with:
Details
This PR intentionally does not add a new
pnpm-config.jsonoption. The setting is controlled by an environment variable so all subspaces use the same behavior for a given install.When
RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1is set, Rush validates that:10.12.1or newer."pnpmStore": "global"orRUSH_PNPM_STORE_PATH.usePnpmSyncForInjectedDependenciesis not enabled, because pnpm-sync currently assumes the virtual store lives undernode_modules/.pnpm.Store selection keeps the existing Rush behavior:
RUSH_PNPM_STORE_PATH, Rush passes that path to PNPM."pnpmStore": "global"and noRUSH_PNPM_STORE_PATH, Rush lets PNPM use its default global store location."pnpmStore": "local"and noRUSH_PNPM_STORE_PATH, enabling global virtual store is rejected because the store would still be worktree-local.If
RUSH_PNPM_STORE_PATHpoints inside the repo, Rush prints a warning instead of failing, since the install can still work but will not reduce setup or cleanup cost across worktrees.Rush also records the global virtual store setting in the last-install flag. If
RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STOREor the effective PNPM store path changes between installs, Rush reports an error and asks forrush install --purge/rush update --purgeinstead of silently mixing install layouts.With PNPM 10.12.1+,
common/temp/node_modules/.pnpmmay still exist, but it only contains lightweight metadata such aslock.yamlandnode_modules. Package instance folders are stored under the shared PNPM store.Testing
Ran the Rush lib test suite:
node common/scripts/install-run-rush.js test --to @microsoft/rush-lib --verboseThis covers environment variable parsing, workspace file generation, validation errors, last-install flag mismatch detection, and PNPM store argument behavior.
Added a PNPM global virtual store integration test that creates a temporary Rush repo, enables the feature with
RUSH_PNPM_ENABLE_GLOBAL_VIRTUAL_STORE=1, setsRUSH_PNPM_STORE_PATH, then runs:The test verifies the generated workspace file, shared PNPM store usage, dependency links, and built output execution.
Also ran:
Result: npm, PNPM global virtual store, and yarn integration tests all passed.
Question
build-tests/rush-package-manager-integration-testpreviously only covered npm and yarn, and its package description referred to "non-pnpm package managers." I added the PNPM case there because this feature needs real install-structure validation. Please confirm whether this is the right place for that coverage.